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Applying Insights from Faculty Teaching Practices 
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Key Points 

1. Teaching practices vary 
considerably by discipline, so 
"STEM" as a catch-all category 
may not make sense in practice. 
Instead, policymakers and 
practitioners should focus on 
discipline-specific practices and 
workplace conditions when 
considering pedagogical reforms. 

2. Pedagogical reform efforts must 
be grounded in more detailed 
accounts of teaching practice, 
which can provide insights 

into the nature of educational 
reform, implicate new ways to 
think about program design, 
identify leverage points for future 
interventions, and contribute 
to program evaluation and 
institutional assessments. 
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By Matthew Tadashi Hora 

Introduction 

Policymakers and educators are increasingly expressing concerns 
that the U.S. is lagging behind other countries — notably China 
and India — in educating the next generation of mathematicians 
and scientists who will create the products and innovations that 
drive the 21st century economy. In addition, concerns about 
the underrepresentation of women and minorities in science 
and math contribute to a rising chorus of critiques of how these 
disciplines are taught in institutions of higher education (IHEs). 

In response, the federal 
government, private foundations, 
and many IHEs are encouraging 
STEM (science, technology, 
engineering, and math) 
instructors to adopt interactive, 
inquiry-based teaching methods 
in their classes, laboratories, and 
discussion sections. For example, 
the Nobel-prize winning physicist 
Carl Weiman recently developed 
a five-year $15 million initiative 
called the Science Education 
Initiative (SEI) at the University 
of British Columbia and the 
University of Colorado at Boulder to help math and science faculty 
develop research-based learning goals and corresponding teaching 
and assessment strategies. 

Many of the strategies for changing teaching practices are based 
on research from the learning sciences that links improved 
student learning and the acquisition of higher order cognitive 
skills with methods such as problem-based learning, as opposed 
to a singular reliance on traditional, didactic lecturing. 1 



...many IHEs are 
encouraging STEM 
instructors to adopt 
interactive, inquiry-based 
teaching methods in 
their classes, laboratories, 
and discussion sections. 
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Research also suggests that one of the 
reasons students drop out of math 
and science majors is the perceived 
poor quality of instruction and lack 
of opportunities to be more actively 
engaged with their instructors and 
the material . 2 Based on this evidence, 
efforts to encourage instructors 
to adopt research-based teaching 
methods are on the rise. 

However, evidence also suggests that 
the adoption of interactive teaching 
techniques is slow. Research on the 
diffusion of innovations reveals 
that structural barriers, entrenched 
cultures, and top-down interventions 
that conflict with local practices can 
contribute to the maintenance of the 
status quo. 

Furthermore, individual instructors 
play a key role in the organizational 
change process because they adopt, 
adapt, or reject new policies and 
innovations based on a combination 
of their own expectations and 
experiences, local traditions, and, 
especially, the multiple demands 
and pressures that shape classroom 
practice. Thus, a major challenge facing 
policymakers and educators engaged 
in pedagogical reform in math and 
science is to fully understand teaching 
practices in IHEs and how instructors 
experience their organizational 
environment in regards to their 
teaching responsibilities. 



An adequate accounting of these 
complex phenomena involves 
capturing many different aspects of 
teaching practice. This includes 

■ moving beyond a 
narrow conception 
of teaching that is 
limited to providing 
lists of specific 
teaching methods 
(e.g., lecture 
and small- group 
discussion) utilized 
during a class ; 3 

■ accounting for 
the influence of 
contextual factors 
in the form of 
artifacts (designed 
entities such as curriculum or 
instructional technology); 

■ documenting aspects of teacher- 
student interactions, including 
the cognitive demand (instructors’ 
expectations of the depth and 
complexity of student thinking) 
placed upon students throughout 
the course of a class; and 

■ understanding the course planning 
process wherein the instructor 
makes decisions about which 
topics to teach, and how, according 
to the perceived constraints 
(curricular pressures, time, etc.) 
that their department or IHE 
places on their teaching practice. 



...structural barriers, 
entrenched cultures, 
and top-down 
interventions... 
contribute to the 
maintenance of the 
status quo. 
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Importantly, these facets of teaching 
do not exist in isolation from one 
another, but constitute parts of a 
larger system-of-practice wherein they 
interact synergistically in real-time. 

However, current approaches to 
studying teaching practices are 
limited in their ability to capture 
these aspects by 

■ reliance on self-reported survey and 
interview data that lack validity; 

■ an overly reductive view of teaching 
as simply the specific teaching 
methods used by instructors; 

■ lack of observation protocols that 
account for multiple dimensions 
of teaching practice; and 

■ limited use of analytic techniques 
to discern the underlying 
patterns within observation data. 

This brief presents an approach 
that addresses these limitations and 
captures the nuances of teaching 
practice with the specific aim of 
providing actionable evidence for 
policymakers and practitioners. 

Methods 

The Culture, Cognition, and Evaluation 
of STEM Higher Education Reform 
(CCHER) project is a National Science 
Foundation-funded, mixed-methods 
study focused on untangling the 
various influences and components 



of teaching practices in IHEs. 4 Our 
approach to analyzing teaching 
practices in real-time is based on 
systems-of-practice theory, which 
accounts for the synergistic interactions 
among actors, artifacts, and tasks that 
collectively constitute an entire activity 
system. 5 In this way, the unit of analysis 
is not the individual instructor, but 
their interactions with the tasks they 
engage in and the artifacts they use 
during a class. 

A key part of the CCHER study 
included developing a new 
observation instrument that captures 
three dimensions of teaching 
practices (teaching techniques, 
cognitive demand, and instructional 
artifacts) called the Teaching 
Dimensions Observation Protocol 
(TDOP). The TDOP includes codes 
representing various teaching 
practices. In this study, analysts 
conducted observations every five 
minutes over the course of a 50- to 
70-minute class period and circled 
TDOP codes corresponding to the 
practices they witnessed. 6 Thus, a 50- 
minute class yields 10 separate sets of 
data that could be populated with a 
variety of individual codes. 

In the spring of 2010, researchers 
observed 57 instructors in five disciplines 
(mathematics, biology, physics, 
chemistry, and earth/space sciences) 
at three research IHEs, including the 
University of Wisconsin-Madison. Each 
instructor was interviewed once and 
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Table 1 



Description of Study Sample 





CCHER Sample 


n 


% 


Level of Course 




Lower division 


39 


68% 


Upper division 


18 


32% 


Sex 




Female 


22 


39% 


Male 


35 


61% 


Discipline 




Math 


18 


32% 


Physics 


11 


19% 


Chemistry 


9 


16% 


Biology 


11 


19% 


Earth/Space science 


8 


14% 




CCHER Sample 


Spring 2010 Instructors 


n 


% 


n 


% 


Position Type 




Lecturer 


29 


51% 


140 


53% 


Assistant Professor 


6 


11% 


24 


9% 


Associate Professor 


4 


7% 


20 


8% 


Professor 


18 


31% 


79 


30% 



observed twice for a total of 109 observed 
classes. 7 Information about the study 
sample is provided in Table 1. 

The observation data were analyzed 
descriptively by identifying the 
percentage of total intervals in which 
a particular code was observed and 
through network analysis procedures, 
which depict the strength and extent of 
interactions among codes. The interview 
data were analyzed using thematic 
analysis procedures to identify perceived 
constraints to teaching. 8 



Findings 

Results from Wave 1 of the CCHER 
study are reported in this brief for 
the entire sample (57 instructors; 

109 classes) as well as for physics and 
math instructors in order to examine 
any disciplinary variations in 
teaching. Findings are grouped into 
three areas: 1) descriptive accounts 
of specific teaching practices, 2) 
inter-connections among teaching 
practices, and 3) perceived 
constraints on teaching. 
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Descriptive Accounts of 
Specific Teaching Practices 

First, data from the classroom 
observations are presented for three 
teaching practices: 1) teaching 
methods, 2) cognitive demands, and 
3) instructional artifacts. These data 
represent the percentage of total 
intervals in which a particular code 
was observed. 



Teaching Methods 

Table 2 depicts the percentage of 
intervals in which a particular 
teaching method was observed 
for given groups. The methods 
were identified through a review 
of the literature, a pilot study, and 
consultations with instructors in 
math and science fields. 



Table 2 

Percentage of Total Intervals in which Specific Teaching Methods Were Observed During Classes 





Entire Sample 

1,178 intervals 
57dasses 


Mathematics 

381 intervals 
18 classes 


Physics 

219 intervals 
11 classes 


Teaching Methods 




Lecture 


83% 


75% 


93% 


Illustration 


12% 


7% 


13% 


Demonstration 


10% 


1% 


40% 


Small group discussion 


3% 


4% 


4% 


Whole class discussion 


2% 


0% 


0% 


Multimedia 


3% 


0% 


7% 


Case study 


2% 


0.1% 


0% 


Worked out problems 


27% 


66% 


18% 


Desk work 


5% 


10% 


1% 


Online techniques 


0% 


1% 


0% 


Brainstorm 


1% 


0% 


1% 


Rhetorical question 


8.5% 


11% 


5% 


Display conceptual question 


22% 


21% 


17% 


Display algorithmic question 


9% 


24% 


3% 


Comprehension question 


13% 


21% 


5% 


Novel question 


6% 


8% 


3% 


Clicker conceptual question 


6% 


0% 


8% 


Clicker algorithmic question 


1% 


0% 


5% 
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The specific teaching methods observed 
for the entire sample indicate a strong 
reliance on the lecture method (83%). For 
secondary methods, instructors worked 
out problems (27%) and posed questions 
to students including conceptual questions 
(e.g., How does Coloumbs law relate to 
motion?) (22%), algorithmic questions 
(e.g., What is the solution for X?) (9%), 
and comprehension questions (e.g., Do 
you understand?) (13%). 

For math and physics instructors, the 
lecture method also was prevalent, 
but with stark differences in relation 
to the use of working out problems, 
demonstrations, and different types 
of questions. It is also important to 
note that particular categories, such as 
the lecture method, were often used 
in conjunction with other methods 
(e.g., illustration, rhetorical question, 
etc.), which is apparent in the network 
analyses. This underscores the danger 
in interpreting teaching practices 
based on reported frequencies of single 



methods, as they frequently co-occur 
with other methods or change rapidly 
from moment to moment. 

Cognitive Demands 

Table 3 depicts the percentage of 
intervals in which a type of cognitive 
demand was observed. These data are 
based on analysts’ interpretations of 
the types of thinking that instructors 
expected of their students. For 
example, the category receive/ 
memorize would be selected if an 
instructor presented facts, concepts, 
or ideas in a way that did not invite a 
student response or active engagement 
with the material. 

The dominant cognitive demand 
observed for the entire sample was 
the receive/memorize category 
(89%), with the secondary type 
being problem-solving (31%), where 
students were expected to follow and 
understand the specific solution paths 
for solving problems. 



Table 3 



Percentage of Total Intervals in which Specific Cognitive Demands were Observed During Classes 





Entire Sample 

1,178 intends 
57 classes 


Mathematics 

381 intends 
18 classes 


Physics 

219 intends 
11 classes 


Cognitive Demands 




Receive/memorize 


89% 


83% 


93% 


Problem solving 


31% 


58% 


28% 


Creating 


8% 


6% 


11% 


Integration 


7% 


7% 


7% 


Connections to real world 


14% 


6% 


24% 
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Table 4 

Percentage of Total Intervals in which Specific Instructional Artifacts were Observed During Classes 





Entire Sample 

1,178 intervals 
57 classes 


Mathematics 

381 irtenals 
18 classes 


Physics 

219 intervals 
11 classes 


Artifacts 




Blackboard 


41% 


75% 


48% 


Laptop/slides 


45% 


0% 


57% 


Handouts 


2% 


0.1% 


0% 


Book 


0% 


1% 


0% 


Demonstration equipment 


7% 


0% 


33% 


Clickers (conceptual question) 


6% 


0% 


8% 


Clickers (algorithmic question) 


1% 


0% 


5% 


Miscellaneous object 


6% 


3% 


11% 


Pointer 


19% 


0% 


9% 


Digital tablet 


5% 


6% 


9% 


Poster 


1% 


0% 


1% 



The receive/memorize category was 
also dominant for math and physics 
instructors, with more emphasis in 
physics than mathematics, where many 
intervals were coded for problem solving. 
This is unsurprising since many math 
instructors were observed working out 
problems on the blackboards during the 
class. Additionally, physics instructors’ 
frequent use of demonstrations and 
lectures interspersed with illustrations 
from the physical world was often 
accompanied by the connections to the 
real world cognitive demand. 

Instructional Artifact Use 

Table 4 depicts the percentage of intervals 
in which a type of artifact was observed. 
The primary artifacts observed for the 



entire sample include laptop computers 
(45%), which were used primarily 
to project PowerPoint slides, and 
blackboards (41%). 

Secondary artifacts included pointers 
(19%), which were generally used to 
indicate key aspects of projected slides 
or solution paths to problems written 
on the board, and clicker response 
systems (6% with conceptual questions 
and 1% with algorithmic questions). 

The data indicate differences between 
disciplines. Mathematics instructors 
primarily used blackboards while 
physics instructors split their artifact use 
among blackboards, laptop/slides, and 
demonstration equipment. 
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Inter-Connections Among 
Teaching Practices 

While descriptions of specific teaching 
practices are revealing, they are only 
fruitfully interpreted as constituent 
parts of a synergistic whole, rather than 
isolated features of teaching practice. 
Systems-of-practice theory argues that 
teaching is a complex act involving the 
simultaneous interactions among these 
different features, and we analyzed these 
data using network analysis techniques 
in order to depict how they interact 
during a class period. The results include 



affiliation network graphs that illustrate 
the strength of the affiliation between two 
particular codes, as represented by the 
relative thickness of the lines. 9 We present 
affiliation network graphs for math and 
physics instructors separately. A graph 
for the entire sample was not intelligible 
given the diverse practices exhibited by 
each disciplinary group. 

The graph for math instructors in 
Figure 1 reveals a tightly connected 
central core that suggests a restricted 
set of overall teaching practices and 
artifact use in particular. 



Figure 1 

Affiliation Network Graph of Observed Codes for Math Instructors 
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Figure 2 

Affiliation Network Graph of Observed Codes for Physics Instructors 




As a group, mathematicians 
demonstrated a primary model of 
instruction that requires students to 
problem-solve and recall/memorize 
information by working out problems 
and lecturing on the blackboard. This 
model is supplemented with a range of 
question styles, including conceptual 
and algorithmic questions. 

The graph for physics instructors in 
Figure 2 reveals a more diffuse central 
core than in the graph for math 
instructors. These physics instructors 
presented a varied model of instruction 



that requires students to problem-solve 
and make connections through the use 
of demonstrations and demonstration 
artifacts. In addition, instructors asked 
students to recall/memorize information 
while they lectured on the blackboard 
and used PowerPoint slides with pointers. 
This primary model was frequently 
supplemented with the use of clickers, 
multimedia, questions, and illustrations. 
The overall model indicates that 
instructors delivered the course material 
through lecturing and demonstrations — 
supplemented by artifacts — that require a 
wide range of cognitive demands. 
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We speculate that the contrast in 
these models reflects differences 
between the forms of knowledge in 
math and physics. In the context of 
the courses included in this study 
(i.e., undergraduate, mostly lower- 
division courses), math operates in 
the realm of ideas and has a very basic 
form largely dependent on symbol 
manipulation to prove theorems and 
other mathematical principles. In 
contrast, physics is grounded more 
in the processes of the physical world 
and has a more applied form, where 
symbols are used but mostly to model 
physical processes. Thus, math has 
basic (i.e., symbolic and non-applied) 
content with a corresponding model 
of instruction, whereas physics has 
applied content modeled through 
applied instructional practices. 

Perceived Constraints on Teaching 

Finally, these accounts of teaching 
practice are incomplete without an 
understanding of the decision-making 
process that leads instructors to teach 
in particular ways. The interview data 
revealed three primary constraints 
that instructors claimed exerted a 
strong influence on their teaching 
practice: 1) course syllabi and other 
departments’ needs, 2) lack of time, 
and 3) student characteristics. 

Course Syllabi 

Respondents noted that course 
syllabi exerted a strong influence on 
the content they focused on and the 



teaching methods they used. Specific 
topics and their sequencing were often 
determined in accordance with other 
departments’ needs, particularly in 
service courses where students from 
other colleges (e.g., engineering) 
took undergraduate courses as part 
of their degree requirements. The 
resulting syllabi frequently include 
a wide range of topics in large part 
because lower level courses build on 
one another such that certain concepts 
are required in order to advance to the 
next course (e.g., calculus sequences). 

As a result, instructors 
felt pressure to cover 
many topics in a short 
amount of time, which 
led to a need to efficiently 
deliver this content via the 
didactic lecture method. 

Additionally, several 
of these courses had 
multiple sections such 
that a common syllabus 
and exam structure was 
in place, which meant 
that instructors could 
not deviate from this 
tight schedule without 
jeopardizing student performance. 

Lack of Time 

One well-known constraint that 
instructors perceive in regards to 
their teaching activities is that of time, 
which is limited given the competing 
demands of their research, service, 



...accounts of 
teaching practice are 
incomplete without an 
understanding of the 
decision-making process 
that leads instructors to 
teach in particular ways. 
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and advising obligations. 
The resulting role stress 
forces instructors to 
choose among these 
different responsibilities. 

For tenure-track faculty 
in particular, research 
activities will sometimes 
take precedence over 
additional preparatory 
time for a class or 
acquiring new teaching skills. Even 
for lecturers who are not subject to 
the pressures of acquiring research 
funding or publishing, they often 
have heavy teaching loads that result 
in a similar lack of time for teaching- 
related activities. 

Student Characteristics 

Respondents also cited certain student 
characteristics as influencing how they 
approached their teaching, including 
student reactions to particular classes 
and teaching methods, and their 
overall level of preparation for the 
course. In several cases, instructors 
“read” their students while teaching the 
class and responded accordingly based 
on their reactions. 

In addition, several respondents observed 
that students were poorly prepared 
for their courses, which necessitated 
reviewing many basic concepts of the 
field as well as using many illustrations 
and examples as a way to “bring to life” 
the topics being discussed. 



...pedagogical reform 
efforts must be 
grounded in more 
detailed accounts of 
teaching practice. 



Implications for 
Policy and Practice 

The findings presented in this brief have 
several implications for policymakers 
and practitioners engaged in math and 
science education in IHEs. First, because 
teaching practices vary considerably by 
discipline, “STEM” as a catch-all category 
may not make sense in practice. Instead, 
policymakers and practitioners would 
do well to focus on discipline-specific 
practices and workplace conditions when 
considering pedagogical reforms. 

Second, given that single-indicator 
accounts of teaching methods do not 
adequately represent the different 
types of practices being used (i.e., 
methods, cognitive demand, and 
artifacts) and how they interact in 
practice, pedagogical reform efforts 
must be grounded in more detailed 
accounts of teaching practice. Detailed 
accounts can contribute to reform 
efforts in the following ways: 

Provide Insights into the 
Nature of Educational Reform 

Research on educational reform 
in K-12 schools demonstrates that 
local practices and traditions will 
influence whether and how teachers 
understand and adopt a particular 
policy or innovation. As a result, 
prior research recommends that 
policymakers and practitioners 
develop a detailed awareness of 
how these local practices influence 
reforms at the classroom level. 10 
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To do so first requires a deep 
understanding of local practices. By 
opening up the proverbial black box 
of teaching practice using a systems - 
of-practice approach, it is possible to 
discern the specific factors that support 
or inhibit reforms and the forces 
guiding their adoption, adaptation, or 
rejection. 

Implicate New Ways to Think 
about Program Design 

Insights into the nature of local 
systems-of-practice can also suggest 
new ways to think about the design 
of educational reforms. For example, 
K-12 reformers focus increasingly 
on the idea that new policies should 
exhibit coherence with existing 
constraints facing teachers — 
particularly obligations regarding state 
standards — rather than conflict with 
them. Researchers argue that new 
products or innovations should be 
tailored to the unique practices and 
workplace conditions of instructors so 
that innovations are actually viable for 
classroom use. 

In addition, by designing programs 
with a grounded understanding of 
local practice, policymakers and 
educators can determine whether 
to support or transform existing 
practices, which is difficult without 
operating from a baseline portrayal 
of local practices. User-based design 
may be particularly important 
for math and science education 



reform in IHEs because research 
suggests that some resistance to 
pedagogical reforms is due to the 
perceived antagonism between 
reform advocates and instructors and 
advocates’ lack of insight into the 
daily work of instructors. 11 

Identify Leverage Points 
for Future Interventions 

In opening up the black box of 
teaching, it becomes possible to 
pinpoint the precise component 
parts of teaching practices in an IHE 
classroom. For instance, in this study, 
instructors consistently cited course 
syllabi as exerting a strong influence 
on how they selected content and 
teaching methods. 

In particular, several 
respondents felt that 
lecturing methods were 
the best way to efficiently 
convey the sheer volume 
of content in many 
introductory courses. 

This suggests that minor 
changes in course syllabi 
may reduce the pressure 
on instructors to cover 
a large number of topics 
and rely on lecturing 
in doing so. With this 
perspective in mind, an intervention 
could pair workshops in interactive 
teaching methods with efforts to 
change course syllabi in lower- 
division courses. 



...using a systems-of- 
practice approach, it 
is possible to discern 
the specific factors 
that support or 
inhibit reform... 
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Inform Program Evaluation and Institutional Assessments 

Finally, as investments increase in changing teaching practices in math and 
science, so too does the desire to identify what works and whether particular 
interventions have met their goals. However, evaluation instruments that solely 
rely on eliciting the teaching methods that instructors use in a particular class are 
not sufficiently sensitive to capture the complexities of teaching practice. 

With the TDOP instrument, it becomes possible to integrate a 
systems-of-practice approach into a variety of evaluation designs, 
including formative evaluations of program implementation and 
summative evaluations that focus on ascertaining changes over time. 
Additionally, these snapshots of practice can be used to provide 
baseline accounts of current conditions for faculty development or 
institutional assessment purposes. 

However, as interest in assessment and accountability rise in higher 
education, many IHEs around the globe are becoming increasingly 
interested in demonstrating the quality of instruction at their 
institutions. At the present time, indicators such as graduation rates 
and student learning outcomes are the most common approach to 
estimating the elusive notion of teaching quality, though researchers 
are actively seeking better metrics. 

It cannot be emphasized enough that the data reported in this brief are not 
intended to be used in this fashion because equating quality with the use (or 
lack thereof) of particular teaching techniques is not a valid or defensible 
proposition. Research suggests that didactic lecturing can be used effectively 
when tied to a broader pedagogical rationale, while interactive methods such 
as clickers can be used ineffectively . 12 As a result, the data reported in this brief 
and the instruments used to collect these data are solely designed to measure 
descriptive accounts of practice. 



...snapshots of practice 
can be used to provide 
baseline accounts of 
current conditions for 
faculty development 
or institutional 
assessment... 
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